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Abstract — The two-user discrete memoryless state-dependent 
multiple-access channel (MAC) models a scenario in which two 
encoders transmit independent messages to a single receiver via 
a MAC whose channel law is governed by the pair of encoders' 
■inputs and by an i.i.d. state random variable. In the cooperative 
state-dependent MAC model it is further assumed that Message 1 
is shared by both encoders whereas Message 2 is known only 
to Encoder 2 - the cognitive transmitter. The capacity of the 
cooperative state-dependent MAC where the realization of the state 
sequence is known non-causally to the cognitive encoder has been 
derived by Somekh-Baruch et. al. 

In this work we dispense of the assumption that Message 1 is 
shared a-priori by both encoders. Instead, we study the case in 
which Encoder 2 cribs causally from Encoder 1. We determine the 
capacity region for both, the case where Encoder 2 cribs strictly 
I causal and the case where Encoder 2 cribs causally from Encoder 1. 

' Index Terms — State-dependent MAC, Gel'fand-Pinsker channel. 
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I. Introduction 



X 



The two-user discrete memoryless state-dependent multiple- 
access channel (MAC) models a scenario in which two encoders 
transmit independent messages to a single receiver via a MAC 
whose channel law is governed by the pair of encoders' inputs 
and by an i.i.d. state random variable S. In the cooperative state- 
dependent MAC model it is further assumed that Message 1 is 
shared by both encoders whereas Message 2 is known only 
to Encoder 2 - the cognitive transmitter. The capacity of the 
cooperative state-dependent MAC where the realization of the 
state sequence is known non-causally to the cognitive encoder 
has been derived by Somekh-Baruch et. al. in UJ. 

In this work we dispense of the assumption that Message 1 
is shared a-priori by both encoders. Instead, we study a "more 
realistic" model in which Encoder 2 "cribs" and learns the 
sequence of channel inputs emitted by Encoder 1 before gener- 
ating its next channel input. Specifically, we study both, the 
case where Encoder 2 cribs strictly causal - i.e. its current 
channel input depends on its message as well as the past inputs 
of Encoder 1 (in the sense of ||2] Situation 2]), and the case 
where Encoder 2 cribs causally - i.e. its current channel input 
depends on its message as well as all past including the current 
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Fig. 1 . State-dependent MAC with a cribbing encoder. 



inputs of Encoder 1 (in the sense of 12^ Situation 3]). The model 
is depicted in Figure 1. 

For both cases - strictly causal cribbing as well as causal 
cribbing - we provide a complete characterization of the ca- 
pacity region. 

The paper is organized as follows. In Section II we provide a 
formal definition for the state-dependent MAC with a cribbing 
encoder. In Section III we present our main results, while 
Section IV is devoted for the proofs. 

II. Channel model 

A discrete memoryless state-dependent multiple-access chan- 
nel is a triple {Xi x A2 x S,p{y\xi,X2,s),y) where Xi and 
X2 are finite sets corresponding to the input alphabets of 
Encoder 1 and Encoder 2 respectively, tS is a finite set cor- 
responding to the alphabet of the state governing the channel 
law, the finite set y is the output alphabet at the receiver, and 
p{-\xi,X2,s) is a collection of probability laws on y indexed 
by the input symbols xi G Xi and X2 G X2 and s E S. The 
channel's law extends to n- tuples according to the memoryless 
law 

n 

Pr{y''\x",X2,s") = Y[piyk\xi,k,X2,k,Sk) , 

k=l 

where xi^k,X2,k, Sfc and yk denote the inputs, state and output 
of the channel at time k, and x^ — (a;i,i, • ■ • , a^i.fc)- 

Encoder 1 sends a message Wi, which is drawn uni- 
formly over the set {1, . . . ,e"^^} — Wi to the receiver, 
while Encoder 2 sends to the receiver a message W2 which 
is independent of Wi and is drawn uniformly over the set 



{1,. 



pn_R,2 



} = VV2. The channel state sequence S", which is 



drawn i.i.d. according to the law ps, is available non-causally to 
Encoder 2. It is further assumed that Encoder 2 "cribs" causally 



and learns the sequence of channel inputs emitted by Encoder 1 
in all past transmissions (in the sense of |2, Situation 2]) before 
generating its next channel input. The model is depicted in 
Figure 1. 

An (e"^^ , e"^^ , n) code for the state-dependent multiple- 
access channel with a strictly causal cribbing encoder consists 
of: 

1) Encoder 1 defined by a deterministic mapping 



/i: Wi^A-i" 



(1) 



which maps a message wi £ Wi to a codeword x" G X" . 
2) Encoder 2 defined by a collection of encoding functions 



/2T^ : W2 X 5" X A-f-i -,X2 k = l,2,...,n 



(2) 



which, based on the message W2 £ W2, the state sequence s" e 
iS" and what was learned from the other encoder by cribbing 



„fc-i 



G X^ , map into the next channel input X2,k G ^2 



3) The receiver decoder defined by a mapping 
g: y"^ Wi X W2 
which maps a received sequence y" to a message pair 

{wi,W2) G Wi X W2. 

An (e"^i , e"^^ , n) code for the state-dependent multiple- 
access channel with a causal cribbing encoder differs from 
that for a strictly causal encoder just in the encoding rule 
at Encoder 2 which is defined by a collection of encoding 
functions 



4i : W2 X 5" X X^ -^X2 fc - 1, 2, . 



(3) 



which, based on the message W2 G W2, the state sequence s" G 
iS" and what was learned from the other encoder by cribbing 
Xi G X^, map into the current channel input X2.k G X2. 

For a given code, the block average probability of error is 



p{n) 



S^ 



^n(Ri+R2) L^ 

uji — 1 u;2 — 1 



Y^ P^"\W,,W2) 



where 



P^"'>iwuW2) = 

Pr{(wi,W2) 7^ {wi,W2)\iwi,W2) sent}. 

A rate-pair (i?i,i?2) is said to be achievable if there exists 
a sequence of {e^^^ , e"'^^ , n) codes with lim„^oo -Fe = 0. 
The capacity region of the state-dependent MAC with a cribbing 
encoder is the closure of the set of achievable rate-pairs. 

III. Main results 

Our first result is a characterization of the capacity region 
for the two-user discrete memoryless state-dependent MAC 
with state-sequence available non-causally at a strictly causal 
cribbing encoder. By combining the coding strategies from yj 
and II2I we prove the following. 



Theorem 1: Consider the discrete memoryless state- 
dependent MAC {Xi X X2 X S,p{y\xi,X2,s),y) with 
state-sequence available non-causally at a strictly causal 
cribbing encoder and finite alphabets S,Xi,X2. The capacity 
region of this channel is 

C= \J Uri,R2): 

PVSUX1X2Y 

0<Ri <H{Xi\V) 

< i?2 < I{U; Y\VXi) - I{U; S\V) 

0<Ri+R2<IiVUXi;Y)-I{U;S\V)\, (4) 



where the union in ^ is over all laws on V^ G V, 5 G 5, f/ G 
U, Xi€Xi,X2eX2,Y ey of the form 



PvsuXiX2y{v, s, u, xi, X2, y) 

= Pv{v)ps{s)pxi\v{xi\v)pux2\sv{u, X2\s, v)p{y\xi,X2,s) 



(5) 



The cardinalities of the auxiliary random variables V and U are 
bounded by 

|V| < 1^-111^-21151+5 

M\ <|A'i||;f2||5||v| + 2. 

Our second result is a characterization of the capacity region 
for the two-user discrete memoryless state-dependent MAC 
with state-sequence available non-causally at a causal cribbing 
encoder 

Theorem 2: Consider the discrete memoryless state- 
dependent MAC {Xi X X2 X S,p{y\xi,X2,s),y) with 
state-sequence available non-causally at a causal cribbing 
encoder and finite alphabets S,Xi,X2. The capacity 
region of this channel is the set of rate pairs satisfying 
(HJi except that the union is taken over all laws on 

V e V,S e S,U e U,Xi e Xi,X2 e X2,Y e y of 

the form 

PVSUXiX2y{v, S, U, Xi,X2,y) 

^Pv{v)ps{s)pxi\v{xi\v)pu\sv{u\s,v) 

Px2\vusxAx2\v,u,s,xi)p{y\xi,X2,s). (6) 



IV. Proofs 
A. Proof of the achievability part in Theorem [7] 

We propose a coding scheme that is based on Block-Markov 
superposition encoding and which combines the coding tech- 
nique of |1| with that of |2|, while the decoder uses backward 
decoding. 



1} Coding Scheme: We consider B blocks, each of n sym- 
bols. A sequence of i? — 1 message pairs [Wi , W2 ), for b = 
1, . . . , B — 1, will be transmitted during B transmission blocks. 
Here the sequence {VFf } is an i.i.d. sequence of uniform 
random variables over {l, . . . , e"^^ } and independent thereof 
{W2 } is an i.i.d. sequence of uniform random variables over 



{1- 



.,e 



nR2 



}. As i? — !• cx), for fixed n, the rate pair of the 



message (M^i.W^a), {RuRi) = {Ri{B-l)/B,R2{B-l)/B), 
is arbitrarily close to (i?i,i?2)- 

We assume a tuple of random variables V^ G V, S* G 5, C/ G 
U,Xi G A'i,X2 G ^-2,^ G y, of joint law Q. 

Random coding and partitioning: In each block 6, 6 = 
1, 2, . . . , i?, we shall use the following code. 

> Generate e^^^ sequences v — (ui,...,w„), each with 
probability Vi {v) ~ Y\j:=iPv{vk)- Label them v {ujq) 



where wq e {l, ■ • • ,e"^i}. 



aflRl 



sequences 



For each v (wq) generate 

Xi = {xi,i,Xi^2, ■ ■ ■ ,xi^n)^ each with probability 
Pr(a;i|ii(wo)) = nfc=iP-^i|v(2^i.fe|ffc(wo))- Label them 



Xi {i,uJo),i G {1, 



..,e 



nfl 



^}- 



For each v (wo) generate e"*^^^+^ •* sequences u = 
{ui,U2, ■ ■ ■ ,Un), each with probability Pr {u\v (ujq)) — 
YVk=iPu\v{'^k\vk{oJoj)- Randomly partition the set {u} 



nR' 



into e" 2 bins, each consisting of e" codewords. Now 
label the codewords by u, (j', j, ojo) , i G {!,..., e"^^ }, j G 
{!,..., e"^ } where j identifies the bin and j the index 
within the bin. 

Encoding : We denote the realizations of the sequences 
{W{ '} and {W2 } by {w\ '} and {wj }. and the realization 
of the state sequence {sf\s'i\ . . . ,S^n^) by s^^\ The code 
builds upon a Block-Markov structure in which the message 
{w\ ,^2 ) is encoded over the successive blocks b and (fe+1) 



such that, Wq 



wf\ for 6= 1,...,B-1. 



„(6)i 



„('')i 



The messages {w\ } and {wj }, 6 = 1, 2, . . . , i? — 1 are 
encoded as follows: 

In block 1 the encoders send 



W-a;2(.(i),4'\l) 



X 



Here, the encoding a;2(s'^ \u'2 ,1^0 ) i^ defined as follows: 
1) Find the typical u{w2 ,]o,ijj\^ ): Search within the bin 



u{w2 j'ji^o ) for the lowest jo G {Ij 



„n_R' 



} such 

that tt(u'2 jJojWq ) is jointly typical with the pair 
(t;(4''^), «('')); denote this jo as jo(s^''', w2''\'^o''')- If 
such Jo is not found or if the state sequence s^**) is non- 
typical an error is declared and ]q{s'^^\w^ \lo\^') = 1. 
2) Generate the codeword a;2(s^''\w2 jWq ) by draw- 
ing its components i.i.d. conditionally on the triple 
[s'^^\u{w2 ,]a,to\))Tv{Lu)^)), where the conditional 
law is induced by ©. 



Suppose that, as a result of cribbing from Encoder 1, before 
the beginning of block h = 2, 3, . . . , i?. Encoder 2 has an 
estimate wi for w\ . Then, in block 6 = 2, 3, . . . , -B — 1, 
the encoders send 

(6) f (6) (b-l)\ 

X^^^=X2{s(''\wi'\^t'^), 



and in block B 



x\ ^Xi{l,w\ ') 



X 



(B) 



X2{S^^\1,^[''^'^) 



Decoding at the receiver: After the reception of block- i? 
the receiver uses backward decoding starting from block B 
downward to block 1 and decodes the messages as follows. 

In block B the receiver looks for w\ such that 

v[w\ '),xi{l,w\ 0,m(1,jo,K '), 

a;2(s(^\i,«^r"'^),y^^)) GA(V^,Xi,t/,X2,y), 



where Jo =Jo(s'^\l>l^ ^^)- 

Next, assume that, decoding backwards up to 
(and including) block & + 1, the receiver decoded 



(B-l) , -(S-1) -(S-2) 



>,{W, 



,w{ 



, ,(m 



[b+l] 



,w{ 



[b), 



To decode 



block b, the receiver looks for (w2 ,Wi ) such that 

/ /-(6-l)\ /-(6) "(6-1)n /-(b) -(b-l)\ 

[y{w\ '),xi[w\>,w\ '),u{w)2' ,jo,w\ '), 

X2is<^'\w^^\wt%y^''^) eMv,x,,u,X2,Y), 

where Jo ^ .]ois'^''\w^2^\w[''^^^). 

Decoding at Encoder 2: To obtain cooperation, after block 
& = 1, 2, . . . , i? — 1, Encoder 2 chooses wl such that 

(v{4'~>),x,{wf\^^i^),x[''>) eA,iV,X,,X,), 

where Cjq ~ w{ was determined at the end of block & — 1 
and Wq = 1. 

When a decoding step either fails to recover a unique index 
(or index pair) which satisfies the decoding rule, or there is 
more than one index (or index pair), then an index (or an index 
pair) is chosen at random. 

2) Bounding the Probability of Error: Genie-aided argu- 
ments as in [3 1 and JU can be used to show that the probability 
that either Endoder 2 makes an encoding error or the receiver 
makes a decoding error after block b in the above scheme 
is upper bounded by the probability that at least one of the 
following events Eq — E^ happens. 

Error events: 



. E, 



(b) 



[viJi\u{wf\jo,Ji\x^{w^^\J^^)) 
^A,{V,U,Xi). 



E[ : There exists wi ^ wi such that 

E2 : There doesn't exist ja £ {1, . . . , e"^ } such that 



^A,{V,U,Xi,X2,Y). 

• E^ : There exists uq ^ ujq such that 

for some pair (j, jo) , J G W2 , jo e {1, ■ • ■ , e"-"'}. 

• -E5 : There exists W2 ^ W2 such that 

eAe{V,U,Xi,X2,Y), 

for some index jo G {1; ■ • ■ j e"^ }■ 
We define the event 

Fi'^^[JEf\ b=l,...,B, 

the event 






(b) 



the event 



F3^UK^u£;f)), 



the event 



i=i 



F,^\j[Ei'''>UE^/''>UE^i'>], 



and the event 



F4 ^ [J (i?^'^ u i?f ^ u i?f u i?; 



We can upper bound the average probabiHty of error Pg 
averaged over all codebooks and all random partitions by 

B 



Pe < ^ ^ Pr 

6=1 



E, 



1 

B 

&=i 



Pr 



^W|^^e^^(l...b-l)' 



Pr 



i^fVa 



Pr 



i?fV4^i?f■■■'~'^' 



(b)|^_^c ^(fc+l...B)' 



b=l 



where F^^-'' 1) denotes the complement of the event F^^^ U 
...UF(''-i). 

Furthermore, we can upper bound each of the summands in 
the last component as 

Pr(Ff'|F4^J^^'■■■''' 



= Pr Ui;f|F/,Pf+^-^)' 
\i=4 

<Pr(£;fV/,Pr'-''^' 

+ Pr(i?f|F/,Fr'-''^' 

In the following we separately examine each of the above 
summands. 

r /L\ /I L 1 \ en 

and 



By Lemma [T] (Appendix A) Pr 



Ei'^\F,^,Ei'-'-'^' 



Pr 



E, 



(b) 



can be made arbitrarily small for sufficiently large n. 
Also, by Lemma 121 
. If 



Ri<H{Xi\V), 



(7) 



E^i^\F2',E, 



{i...b~iy 



then Pr 

small, provided that n is sufficiently large; 

If 



can be made arbitrarily 



R1+R2 + R' <I{VUXi;Y), (8) 

('£;f^|F4^pf+^■■■^^''] can be made arbitrarily 
small, provided that n is sufficiently large; 
If 



then Pr £;^"'|P4^Pl 



R2 + R' <I{U:Y\VXi) 



Wipe p(b+l...S)< 



(9) 



can be made arbitrarily 
small, provided that n is sufficiently large; 
Finally, by the covering lemma (See ISl, 13, ID or 191 Chapter 

13]), if 



R' >IiU;S\V) 



(10) 



p(^' I p 



i=i 



then Pr E'j^'lFs'^ can be made arbitrarily small, provided that 
n is sufficiently large. 

The combination of Q, (El, (JU, and ( fTOl i establishes the 
achievability of the rate region (JUi for a law of the form (JS}. 



B. Proof of the converse in Theorem |7] 

Consider an (e"^i , e"^^ , n) code with average block error 
probability P^'^\ and a law on Wi x W2 x X{^ x X^' x 3;" x 5" 
given by 

n 



fc=l 



Let Vk be the random variable defined by 
and let Uk be the random variable defined by 



(11) 



(12) 



(13) 



We start with an upper bound on Ri by following similar 
steps as in ||2l Section V — Converse for situation 2]. 

nRi = H{Wi\W2) 

< I{Wi;Y-''\W2) + n5{P,) 

n 

= ;^/(Xi,fe;y"|W^2^^i)+n5(Pe) 
fc=i 



k=l 

n 

= Y,H{Xi^k\Vk)+nS{Pe). 



n 

^^ ^ [l{W2Y''-^S^^+,;Yk\WiXt'X^,k) 

k=l 

-/(W^2r'=-'^fc+i; SklWiXt')] + nS{P,) 

n 

^^ ^ [l{W2Y'-^S^^+,;Yk\Xt'X^,,) 

k=l 

-I{W2Y''-'S^+^; SklWiXt')] + nSiPe) 

n 

'^' ^ [l{W2Y'~^S^^^,;Yk\Xt'X,,k) 
fc=i 

-I{W2Y''-'S^+^;Sk\X'[-')] +nS{P,) 

n 

= Y,iIiUk;Yk\VkXi,k) ~ I{Uk; Sk\Vk)] . (15) 

fc=i 

Here, 

(6) follows by the Csiszar-Korner's identity ['5! Lemma 7]; 

(c) follows since {W2SJ^_^^^) is independent of Sk', 

(d) follows by the encoding relation ([U; 

(e) follows since Wi<^ Xi^kX'[''^<^ W2YkY''-^S]^^^ and 
W^i-& Xi^fcX^^^^^ Yfe are Markov strings; and 

(/) follows since Wi^ X'[~^<^ W2SkY'''^SJ^^i is a Mai'kov 
string. 

Finally, we consider the sum-rate i?i + R2 

n{R^+R2) = H{WiW2) 

< I{WiW2;Y'')+n6{Pe) 



J2HWiW2;Yk\Y''-^) + nS{P, 



(14) 



fe=i 



fe=i 

where (a) follows from the encoding relation ([T]l. 
Next, consider R2 

nR2 = H{W2\Wi) 

< IiW2;Y^\Wi)+n6{Pe) 



- J2HW2;Yk\WiY^-') + nS{P, 
fc=i 

n 

< Y,HW2Y''-^;Yk\Wi) + nS{Pe 

k=l 

n 

= Y.[l{W2Y'-^Sj:+,;Yk\W^) 



k=l 



-I{S'^+,;Yk\WiW2Y''-^)] +nS{P,) 

n 



<X^[/(W-iW-2r'=-i5^+i;yfe) 
fe=l 

-IiWiW2Y''-'S^+^;Sk)] +nSiPe) 

n 

= J2 [l{W,Xt'X,^kW2Y'^-^Si:+,;Yk) 

k=l 

~IiWiW2Y''-^S^+^;Sk)] +ndiPe) 

n 

t^ J2 [l{Xt'X,,kW2Y'^~^S^+,;Yk) 

k=l 

-I{WiW2Y^-'Sl\^;Sk)] +nS{Pe) 

n 

= ^ [I{VkUkXi,k; Yk) - I{Wi; Sk) 
fe=i 

-IiW2Y''~'S'^+^;Sk\Wi)] +n5(Pe) 

n 

= J2 [HVkUkXix, Yk) - I{Wi; Sk) 



k=l 



fe=l 



I{Y''-';Sk\WiW2Sj:+,)] +nS{Pe) 



fc-lx 



/(W^2>^"~'5^+i; Sk\WiX^-')\ + nS{Pe) 



(c) 



^=^^[/(w^2r'=-i5,V;n.|M^i) 



{^) 



Y, [I{VkUkXix,Yk)~I{Uk;Sk\Vk)] . (16) 



fc=i 



fc=i 



Here, 

{g) follows by the same procedure as (b) and (c); 

(/i) follows by the encoding relation ^ and since 

Wi<^ Xi,kX'l-'^W2Y''-^SJ^^-^^<^Yk is a Mai'kov string; 
and 
(?) follows since Wi is independent of Sk and since 
Wi^ X'[-'^W2Y''-^SJ^^j^<^ Sk and Wi<^ X^-^<^ Sk are 
Markov strings. 

Next we verify the joint law of the auxiliary random variables. 
By ( fTTI ) and the encoding rule (|2]i we may write 

PWiW2X'^-^Xi,kS''-^SkS^^-,X^Y''-^ = 

'PW2Px^\W2X'[-^ S"Pyi'-i-\x'[-^ x^-^ S''-'^ 
Summing this joint law over wi we obtain 

= Pw2X^-^Xi,kS>'-^SkSj;^^x^Y''-^ 
= Px',-'Px,,,\x',-'Ps''-^Ps,Ps'^^, 

Summing this joint law over all possible sub-sequences 
(si, S2, • • ■ , s/c-i) we obtain 

Z^ Pw2Xl-^Xi,kS''-^SkS^_^_,XlYi'-^ 

{si,S2,...,Sk-l) 

=Pxri-Pxi,,ixrips.Ps^vi 

•PH'2Pxf|H'2xJ^-^SfcS;^^iPy''-MxJ^-ixJ-^ 
This establishes the Markov relation 

X2MUk<^ SkVk<^ X^^k- (17) 

Next, let J be a rv. uniformly distributed over {1, . . . , n} and 

independent of {Xi^k,X2M,Vk, Uk, Sk, Yk) , fc = 1, . . . , n, and 
define 

iS,Xi,X2,V,U,Y) ^ {Sj,X,,j,X2j,Vj,Uj,Yj). 

We may express (fT4] i as follows 

1 " 
i?i < -^if(^i,fe|14) = i^(^i|t^, J) = i^(^i|T/),(18) 

fc=i 

where in the last step we've defined V = {V, J). 
Similarly, we may express dTsT l as follows 

1 " 

i?2 < -Y.i^i'^k;Yk\VkXi^k) - I{Uk;Sk\Vk)] 



Finally, we may express (fTST i as follows 

Ri + R2< -y^[I{VkUkX,y,Yk) - I{Uk;Sk\Vk)] 

k=l 

^ I{V,U,XuY\J) ~ IiU:S\V,J) 

= I{V, J, U, X,;Y)^ I{J: Y) - I{U; S\V, J) 

< I(y, J, U, Xi;Y) - I{U: S\V, J) 

= IiV,U,Xi;Y)-IiU;S\V). (20) 

This establishes the single letter expression for the achievable 
rate region (01). The convexity of the rate region (JUl can be 
shown in a similar way. 

The inequalities (fT4b . ( fTSl l. ( fT6l ) combined with their re- 
spective single-letter expressions and the Markov relation (fTTb 
establish the converse part of Theorem 1 . 



C. Bounds on alphabets sizes in Theorem\l\ 

We consider the alphabet sizes of U and V . Specifically, let 
Pxi,X2.s,v.u be a distribution satisfying the Markov conditions 
required in Q. For convenience, Pxx,X2,S,u\v{xi,X2tS,u\v) 
will be denoted in the sequel as P{-\v). We would like to bound 
the sizes of the alphabets V and U, while preserving the region 
given in (|4|. For a generic distribution Q on Xi x X2'x S xU, 
define the functionals 

qxi,X2AQ) = '^Q{XI,X2,S,U), Xi,X2,S e XiX X2XS 

(21a) 



Ji{Q)= X! Q{xi,X2,s,u)\og 



J2{Q)= ^ Q{xi,X2,s,u)\og 

X\ ,X2 ,S,U 



(21b) 



1 



(21c) 



V^ J2x' x' s'Qi^'l'^2J^'^'U') 

MQ)^ 2_^ Qixi,X2,S,u)\0g '' "' , ; — 

xi,X2,s,u 2^x[,x'^^{^1^^2iS,U) 

(21d) 

J4{Q) = ^ Q{xi,X2,S,u) 



Xi ,X2 ,S.U 



•log 



T,x's',u' Q{X1,X'2,S', u')PY\XuX2,s{y\xi,x'2,s') 



(21e) 



fc=i 



Jb{Q)^ X! Q{xi,X2,S,u) 

1 



Xl ,X2,S,U 



I{U;Y\V,Xi,J)-I{U;S\V,J) 
I{U;Y\V,X,)-I{U;S\V), 



•Ids 



J2x',s' Qi^l^ ^21 S' ,u)PY\Xi,X2.,s{y\xi,x'2, S') 



(19) 



(21f) 



Substituting the distribution Pxi,X2.s,u\v{'\^) i" the function- 
als, and averaging them with respect to v, we obtain 



>_^Pv{v)q,,^x,APi-\^))-- 


= Pxi,X2,s{xi,X2, 


s) (22a) 


J2Pviv)MPi-\v))-- 


= H{Xi\V) 


(22b) 


)_^Pviv)J2iP{-\v)). 


= HiS\V) 


(22c) 


Y,Pv{v)MP{-\v))-- 


= H{S\U,V) 


(22d) 


)_^Pv{v).h{P{-\v)). 


^H{Y\V,X,) 


(22e) 


Y,Pv{v)J5{P{-\v))-- 


= H{Y\V,U,Xi). 


(22f) 



Observe that preserving the values of the right hand sides of 
(I22al l- (l22fb . guarantees that we also preserve the region (|4]l. 
We used here the Markov structure F^ (Xi,X2, S)<^ {V, U), 
and the fact that if we preserve the joint distribution of 
Xi , X2 , S, the distribution of Y is also preserved. By the 
Support Lemma 1101 . we can restrict the alphabet of V to: 



|V| < \Xi\\X2\\S\ + 5. 



(23) 



Note that this bound is independent of the alphabet of U. 

We now fix some V with bounded alphabet as above, and 
proceed to bound the alphabet of U. Let Q be a generic 
distribution on Xi x X2 >^ S x V, and define the functional 



qxi,X2,s,viQ) = Q{xi,X2,S,v) 
JliQ) = X! Qixi,X2,S,v) 

•log ^^ — 

T,x[,x'^Qix'l,x'2,S,v) 

hiQ) = ^ Q{xi,X2,s,v) 



(24a) 



Pxi.X2,s.v\u{'W) in the functionals (|24] | and averaging over u, 
we obtain 

X!^C/(w)'7xi,2;2,s,t,(i^(») = Pxt,X2,sy{xi,x2,s,v) 

(25a) 
Y,Pu{u)Ji{P{-\u))=H{S\V,U) (25b) 

U 

Y,Pu{u)J2{P{-\u)) ^ H{Y\V,U,Xi) (25c) 

u 

Applying again the Support Lemma, we see that it suffices to 
bound the alphabe size of U as 

1^1 < |'^i||.^2||'5||V|+2. (26) 

This completes the proof of the bounds on the alphabet sizes. 

D. Proof of Theorem |2] 

The achievability part follows similarly to that of The- 
orem [T] the only difference being in the way the code- 
word X2{s^^\w2 ,ijjQ ) is generated. Here the second en- 
coder generates the codeword 0:2(5'^^', iuj^Wq) by draw- 
ing its components i.i.d. conditionally on the quadruple 
{s^^\u{'W2 ,Jq,0Jq),v{ujq'),x\'), where the conditional 
law is induced by dSJ. 

For the converse, consider an {e"^'^ , e"^'^ , n) code with 

(n) 

average block error probability Pe , and a law on Wi x W2 x 

X^" X X^ X J" X 5" given by 

VWxW^X^X^S^Y'^ 

n 

= PWiPW2l{x^=fi{Wi)]Px:,-\WiW2S^ \\PYk\Xt,kX2,kSk- 



fe=l 



(27) 



(24b) 



X\ .X2 ■,S,V 



•log 



'Ex's' Q{xiiX'2,s', v)PY\XuX2,siy\xi,x'2,s') 



(24c) 



Since 



PY\V,Xi{y\v,Xi) = '^PY\Xi,X2,s{y\xi,X2,s) 

X2,S 

PxuX2,s,v{xi,X2,s,v) 



Ex'^.s' Pxi,X2,S,v{Xl,x'2, S', v) ' 



in order to preserve the value of H{Y\V,Xi), it suffices to 
preserve the joint distribution of Xi, X2, S, V. 

For convenience, we use in the sequel the shorthand nota- 
tion P{-\u) = Pxi,X2,s,v\u{'\'U'). Substituting the distribution 



The Fano inequalities for the causal cribbing case yield the 
same inequalities (fT4l i. ( fTsl l. and ( fT6] l. 

It remains to verify the joint law of the auxiliary random 
variables. 

By dZTJ l and the encoding rule (O we may write 

PWiW2X'^-''Xi^^S''-'>-SkSl^^-^X^Yi'-^ = 

PWiPx^-^\WiPx,,,\WiX'^-^PS''-^PSkPSj:^, 

■PW2Px^-^\W2X';;-^S"Px2,k\W2X'^-^Xi^kX!^-^S" 

'Py''-i-\x^-^x^-^s''-'^ 
Summing this joint law over wi we obtain 

= -PTV2xJ'-iXi.fcS'=-iSfcS^^iXf y*--! 

■PW2Px!;-'-\W2X'^-''S'-Px2,k\W2X^-^Xi,kX^~''S^ 

'Py''-i-\x^-'^xI^-'^s''-'^ 



Summing this joint law over all possible sub-sequences 
(si,S2,---,Sfc-i) we obtain 

(si,S2,...,Sfc_i) 

^Px1-^Px^.u\x^,-^Ps.Psi^, 

■PW2Px^-^\W2Xl-^SkS-^^Px2^k\W2Xl-^Xi^kX!^-''S^Sl^^ 

•Pyfc-l|j^fc-lj^fc-l 

This establishes the Markov relation 



Uk^ SkVk<^ Xi,k, 



(28) 



as well as the fact that conditionally on VkUkSkXi^k the r.v. 
X2^k is independent of the rest. 

Appendix 
A. Strong Typicality 

Let {X(1),X(2\. . .,X('')} denote a finite collection 
of discrete random variables with some joint distribution 
P(:EW,a;(2),...,a;W) with {x^^\x(^\ . . . ,x^''^) G X^^'> x 
A'(^) X ... X X^''\ Let S denote an ordered nonempty subset 
of these random variables and consider n independent copies 
of S. Thus, with 5' ^ (51,52, . . . , 5„), 

n 

Pr{S = s} = l[Pr{S,=s,}. 

Let N{s; s) be the number of indices j e {1,2,..., n} such 
that Sj = s. By the law of large numbers, for any subset S of 
random variables and for all s e S, 
1 



where HA:"!! is the cardinality of the set X. 
Let Ae{S) be defined similar to Ae but now with constraints 
corresponding to all nonempty subsets of S. We recall now two 
basic lemmas (for the proofs we refer to ID). 

Lemma 1: For any e > the following statements hold for 
every integer n > 1: 

1) If s e A(5), then exp (-n(i7(5) + e)) < Pt{S = 
s} <exp(-n(i?(5)-e)). 

2) If 5i,52 C {Xi,X2,...,Xk} and (si,S2) £ A(5i U 
52), then 

exp(-n(iJ(5i|52) + 2e)) < Pr{5i = Si|52 = Sa} 
<exp(-n(iJ(5i|52)-2e)). 

Moreover, the following statements hold for every suffi- 
ciently large n: 

3) Pr{A(5)} > 1-e, 

4) (1 - e) exp(n(ff (5) - e)) < || A(5)|| < exp(n(iJ(5) + 

Lemma 2: Let the discrete random variables X, Y, Z have 
joint distribution Px,Y,z{x,y, z). Let X' and Y' be condition- 
ally independent given Z, with the marginal laws 



P 



X'\Z 



{x\z) = 



V 



Px,Y,z{x,y,z)/Pz{z), 



PY'\z{y\z) 



Y.Px,Y.z{x,y.z)/Pz{z). 



-N{s-s)^P{s), 



(29) 



as well as 
1 



n 



■lnP(si,S2,. 



1 " 

-y^\nP{s,) ^ H{S). (30) 



J = l 



The convergence in ( 1291 ) and ( I30b takes place simultaneously 
with probability one for all nonempty subsets 5 |9|. 

Definition 1: The set Ac of e-strongly typical n-sequences is 
defined by (see W^ Chapter 3,12,13]) 



A = A,(X^^\X'^^\...,X 



^(1) T.(2) 



,X 



(fe) 



(fe) 



T,W.^(1) ^(2) 






.(fc) 



< 



||A:'(i) X ^-(2) X ... xX(^)\y 



Let (X, y, Z) ~ riLi PxxA^k.Vk, Zk) and (X', F', Z) ^ 
riLi -Px'|z(a;fckfc)iV'|z(yfc|2fc)^z(^fc)- Then 
Pr{(X',r',Z) e A(X,r,Z)} <exp(-n[/(X;r|Z) -e]). 
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